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ABSTRACT 

Outcomes of a longitudinal study of the Earn and 
Learn program are summarized. Earn and Learn was a work-experience 
program initiated in 1972 as a school-based, goal-oriented program 
for at-risk middle school students designed to increase their 
self-image and school performance. Two evaluation studies were 
carried out. One study focused on the effects of this program on 
student middle school performance. A second study investigated 
long-term effects of Earn and Learn on high school performance. Both 
studies used a pret es t-pos t t es t experimental-control group design, 
with 44 Earn and Learn students and 35 controls, and 61 Earn and 
Learn students and 51 controls, respectively. Parent feedback was 
also sought. Overall, no differences were found between Earn and 
Learn and control groups for achievement test data, report card 
grades, tardies, suspensions, and graduation rate. Compared to 
controls, Earn and Learn students had fewer middle school absences 
and higher grades in language arts and mathematics (although not 
significantly different); and in high school, Earn and Learn students 
earned significantly more credit hours. Parents generally responded 
favorably to most aspects of the program. As a result of the 
evaluation, the program was restructured with more explicit selection 
criteria and redesigned work experience, community service, and 
career awareness components. Eighteen tables present study findings. 
Appendixes contain criteria for selection, a referral form, and a 
fact sheet on the high school grading system. (SLD) 
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EXECUTIVE SUMMARY 



Objective 

This study summarizes the outcomes of a longitudinal study of the Earn and 
Learn Program, a work experience program for at-risk adolescents. Initiated in 
1972 , the Earn and Learn Program was a school -based, goal-oriented work 
experience program designed to encourage at-risk middle school students to 
develop a positive self image and improve school performance. Enrolled 
students were motivated to iirprove their school attendance, behavior, classwork 
and homework. Earn and Learn staff members worked cooperatively with students, 
teachers, and parents to accomplish these goals. Although earning money was 
not the purpose of the program, it was the incentive used to motivate students 
to achieve these goals. It relied on the support of the local school board, 
businesses, government, agencies, and the community it served. 

In its early years, initial external studies of the program's short term 
effects shewed it to be successful in improving behavior, attendance, and 
grades. Since the 1970* s, however, the program was monitored internally by its 
director. The program continued to purport to be beneficial to students, 
demonstrating behavioral change as well as shewing effectiveness as a drop-out 
prevention program. Parental concerns about the program's claims resulted in a 
request on the part of the local board of education for a formal evaluation of 
the effects of the program. Of particular concern were claims that the program 
was not serving the student population it was intended for; that is, students 
demonstrating poor academic performance, poor attendance or behavior problems. 

Perspective 

In recent years, much attention in the educational arena has been focused on 
the nation's at-risk adolescent youth. These are students who are chronically 
truant, suspended, retained, exhibit poor academic performance or misbehave in 
school. Research indicates that these types of students have a higher-than- 
average chance of dropping out. A great deal of funds have been directed 
toward programs designed to address the needs of such students. For example, 
in Illinois, special funds have been .set aside for Truants Alternatives 
Programs — a funding program that developed out of the state's educational 
reform movement in 1985. Despite these funding efforts, little evidence of 
program effectiveness is available, particularly for drop-out prevention 
programs, because few of these programs have been evaluated formally. 

Methods 

TWo studies were carried out to evaluate the Earn and Learn Program. One study 
focused on the effects of this program on students' performnce in middle 
school. The second study investigated long-term effects of Earn and Learn on 
students' high school performance. Both studies were based on a pretest - 
posttest experimental/control group design. Control groups consisted of 
students who were referred to Earn and Learn, but who either chose not to 
attend or for whom there were no program slots available. Populations were 
equated in terms of sex, race, and academic skills. Study one included 44 
Earn and Learn students and 35 control students. Study two included 61 Earn 
and Learn students and 51 controls. 



Data Source 



Variables under study included standardized achievement test data, report card 
grades, aosences. tardies , suspensions, credit hours and graduation rates. 
Parent feedback was also sougnt. 

Results 

Overall, for most variables under study, no differences were found between Earn 
and Learn and control groups for achievement test data, report card grades, 
tardies. suspensions and graduation rate. Major outcomes include: 

- A significant proportion of Earn and Learn students had fewer absences 
than control students at the end of eighth grade. 

- Earn and Learn students tended to get higher grades in language arts 
and mathematics in eighth grade than did a comparison group of control 
students but were not significantly different. 

In high school, Earn and Learn students earned significantly more 
credit hours at the time of graduation than control students. 

- Overall , parents responded positively to most aspects of the Earn and 
Learn Program. Responses were least favorable with respect to 
communication between staff and students and the summer camp 
experience. 

Educational Importance of the Study 

Earn and Learnt stated purpose is to improve school performance by motivating 
students to improve school attendance, behavior, classwork and homework. Study 
findings suggest that Earn and Learn improved student attendance in the middle 
school years, but had little effect in high school. Although one of the 
program goals is also to improve behavior, Earn and Learn students did not 
enter with severe behavior problems that would be cause for suspension. The 
data at sixth grade indicated that only one student had been suspended in sixth 
grade prior to program study. 

Of particular importance is whether Earn and Learn ! s focus on attendance, 
behavior, homework and classwork contributes % to improved academic performance. 
Study results indicate that students earned'ifiore credit hours and tended to 
perform higher academically than controls; however, there appears to be no 
significant effect on achievement test scores, grades or GPA. Finally, the 
Earn and Learn Program does not appear to improve graduation rate or, 
conversely, decrease the number of dropouts as compared to control groups of 
students. Although Earn and Learn students enter the program with some 
characteristics associated with dropouts, their attendance rate is quite good, 
suspension rate is low and many students perform at above average levels. 

As a result of these findings, the program has been restructured with more 
explicit selection criteria and procedures as well as a redesigned work 
experience and new community service and career awareness components. 
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WORK EXPERIENCE PROGRAMS FOR 
AT-RISK ADOLESCENTS: A COMPREHENSIVE EVALUATION OF 

"EARN AND LEARN" 



PERSPECTIVE 

In recent years, much attention in the educational arena has been focused on 
the nation's at-risk adolescent youth. These are students who are chronically 
truant, suspended, retained, exhibit poor academic performance or misbehave in 
school. Research indicates that these types of students have a higher-than- 
average chance of dropping out. A great deal of funds have been directed 
toward programs designed to address the needs of such students. For example, 
in Illinois, special funds have been set aside tor Truants Alternatives 
Programs — a funding program that developed out of the state's educational 
reform movement in 1985. Despite these funding efforts, little evidence of 
program effectiveness is available, particularly for drop-out prevention 
programs, because few of these programs have been evaluated formally. 

OBJECTIVE 

This study summarizes the outcomes of a longitudinal study of the Earn and 
Learn Program, a work experience program for at-risk adolescents. Initiated in 
1972, the Earn and Learn Program was a school -based, goal-oriented work 
experience program designed to encourage at-risk middle school students to 
develop a positive self-image and iirprove school performance. Enrolled 
students were motivated to improve their school attendance, behavior, classwork 
and homework. Earn and Learn staff members worked cooperatively with students, 
teachers, and parents to accomplish these goals. Although earning money was 
not the purpose of the program, it was the incentive used to motivate students 
to achieve these goals. It relied on the support of the local school board, 
businesses, government, agencies, and the cctmunity it served. 

In its early years, initial external studies of the program's short term 
effects showed it to be successful in inproving behavior, attendance, and 
grades. Since the 1970' s, however, the program was monitored internally by its 
director. Ttie program continued to purport to be beneficial to students, 
demonstrating behavioral change as well as shewing effectiveness as a drop-out 
prevention program. Parental concerns about the program's claims resulted in a 
request on the part of the local board of education for a formal, evaluation of 
the effects of the program. Of particular concern were claims that the program 
was not serving the student population it was intended for; that is, students 
demonstrating poor academic performance, poor attendance or behavior profc^^ms. 

METHODS 

TWo studies were carried out to evaluate the effects of the Earn and Learn 
Program. One study focused on the effects of this program on student's 
performance in middle school. A second study investigated long-term effects of 
the Earn and Learn Program on high school performance. Both studies are based 
on a pre/post experimental-control group design. In addition, parent feedback 
was solicited through the administration of a survey. 



EARN AND LEARN PROGRAM DESCRIPTION 

Elements of the program included: 

a twelve-day suirmer camp experience offering a variety of 
activities such as camping, riding, swimming, canoeing, biking and 
field trips; 

instruction in developing or improving problem-solving cannunication 
and stress rranagement skills; 

practice in "goal-setting; 11 

group counseling; 

an optional half-day summer employment program; 

a summer transition program and weekend camp experience for 
graduating eighth grade students entering high school. 

The ten-month work experience included: 

one hour and thirty minutes of work each day from 3:30 - 5:00 P.M.; 

a pay scale of $3.00 per hour;; 

counseling in development of appropriate work attitudes and 
and behavior; 

daily free transportation from school to the work site and a bus 
route at the end of the work day; 

frequent feedback to parents via phone, home visits, conferences 
and written progress reports. 

Parents were provided the following services: 

daily phone calls from school advisors regarding homework 
completion 

contact from Earn and Learn staff regarding work attendance and 
performance 

group parent education meetings 
social work services 
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In recent years, there were 5.6 budgeted PTE staff members, including a 
coordinator, an assistant coordinator, an aide, a secretary, a .6 social 
worker a .5 bus driver, and a .5 workshop supervisor. In addition, there were 
four advisors located at each middle school who worked one hour daily. 

The program provided a supervised after-school job environment where the 
students earned the privilege of working for the amount of time calculated 
according to their performance on goals during the school day. Goals were 
established during the summer by each student for classwork, homework and 
behavior. After school began, these goals were reviewed and revised in 
cooperation with teachers. Earn and Learn students carried goal cards to class 
each day so that teachers could indicate whether or not the goals were met. At 
the end of the day, students presented the goal card to the Earn and Learn 
school advisor. An analysis of payroll records indicated that students worked 
on the average of four and one-half hours per week. 

Historically, the Earn and Learn Program included summer camp and work 
experience. A homework lab was offered in more recent years. The homework lab 
was provided by staff at the work site. 

Referral Process 

Students were referred to Earn and Learn by middle school staff if they felt 
that student behavior, school attendance, social interaction and academic 
performance could be improved by involvement in the Earn and Learn program (see 
Appendix A for criteria) . A standard referral form was used for each referred 
child. This form (see Appendix B) shows a list of behaviors that teachers must 
consider when referring a student to the Earn and Learn program. These 
behaviors include incomplete homework or classwork, tardiness, non- 
participation in class, withdrawal, inappropriate conduct and aggressiveness. 

In the final analysis, it was the middle school advisor, with input from staff, 
who determined Earn and Learn participants; however, the mnner in which 
referrals were solicited varied from school to school. For exanple, sometimes 
a middle school had an all-school assembly to explain the Earn and Learn 
Program to students. Students were asked to apply if they were interested. 
Teachers were also asked to irake referrals. An interested student must also 
have been identified by middle school staff. Sometimes a school notified 
students of the program via the school newsletter as well as solicited 
referrals from teachers. The, more teachers recommending a child, the more this 
child was given priority status as a candidate. However, middle school staff 
also must have identified these students as likely candidates. Once advisors 
reviewed referrals and identified candidates, students were requested to 
complete an application form and attend an interview at the work site with 
their parents. During this interview, program goals and expectations ware 
discussed and a contract was signed by both parent and student. 

The referral process chaxyxi over tha years. At times, the process required 
the support staff at each school to convene and make decisions about 
referrals. However, this component was not required in recent years, since it 
was found that sometimes these staff members did not always knew the students 
that were referred. For the most part, incoming 7th and 8th graders were 
considered for the program. A few incoming sixth graders were enrolled. 
Participants were given the option of continuing through ninth grade. 



EFFECTS OF THE EARN AND LEARN PROGRAM ON STUDENTS' MIDDLE SCHOOL EXPERIENCE 
The Earn and Learn Student Target Population 

The population for this substudy was 44 Earn and Learn students who graduated 
from eighth grade either in 1988-89 or 1989-90. Students either were enrolled 
in Earn and Learn beginning in seventh grade or in eighth grade. Forty eight 
percent of this group were enrolled in Earn and Learn for both seventh and 
eighth grade. 

Of this population, 24% were white boys, 5% were white girls, 39% were black 
boys and 32% were black girls. This group included students from all four 
middle schools in the district. Forty percent of the students were receiving 
some kind of special education service. In order to determine their entering 
characteristics prior to Earn and Learn intervention, sixth grade demographic 
data were analyzed. The average absence rate of this group at sixth grade was 
13.5 half days per student. This rate is less than the average sixth grade 
absence rate which is 18.5 half days per year. 

The average number of tardies for this group was 13 tardies per year, per 
student* Only one student of this group was suspended from school during the 
sixth grade year. 

The following table shows the stanine distribution of sixth grade California 
Achievement Test Scores for these students in reading, English language and 
math for whcm scores vrere available. 

TABLE 1 Percent of Earn and Learn Students Scoring in Performance 
Groupings: Sixth Grade 

Below Average 
Stanlnes 1-3 

N % 

Reading 9 28 (13%) 

Enqlish Language 13 37 (16%) 

Itothematics 11 31 (14%) 



Average Above Average 

Stanines 4-6 Stanines 7-9 



N 


% 




N 


% 


22 


68 


(50%) 


1 


3 


21 


60 


(48%) 


1 


3 


22 


63 


(53%) 


2 


6 



(37%) 
(36%) 
(33%) 



Th ese figures indicate that approximately one-third of the Earn and Learn 
students performed below average as measured by a standardized achievement 
test. The distribution of the sixth grade population as a whole for these 
tests is shown in parentheses. There are between two to three times more Earn 
and Learn students in the below average group than the total population and far 
fewer in the above average group. 

Average grade point average (GPA) was also ccqputed for Earn and Learn students 
in their sixth grade year. GPA was computed by averaging three marking period 
grades for each child. Based on a scale where "0" represents a failing mark 
and 1, 2, 3, and 4 represents a M D," "C, M "B," and "A" respectively, the 
average reading GPA was 1.93 (C), the average math GPA was 2.51 (C+), the 
average language arts GPA was 2.07 (C), the average science GPA was 1.93 (C) 
and the average social studies GPA was 1.64 (EH-/C-) . 
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Sane of the literature about the Earn and Learn Program described this program 
as having been established ,f to identify those children at an early age who 
manifest school adjustment problems camonly linked with high school dropouts." 
Since the Earn and Learn student population was characterized as manifesting 
behaviors associated with potential high school dropouts, research about 
dropouts was reviewed to determine characteristics having a correlation with 
dropping out of school. 

These characteristics include: 

- low basic skills test performance 

- low grade point average 

- attendance/truancy problem 

- behavior/discipline problem 

- one or more years older than other students in the same grade 

- special program placement 

- minority status 

Given these characteristics , the Earn and Learn student population matched 
some, but not all, of these factors. For example, Earn and Learn students 
generally had a low GPA and approximately one- third tested below average. A 
large proportion (40%) received special services and 36% were 1-2 years older 
than other students in the same grade. However, the average attendance rate 
was better than the district average and suspensions were not a problem. The 
majority of students fell in the average range on standardized tests* 

Other frequently cited characteristics in the research include low socio- 
economic status, unstable heme, abuse, lew parent educational level and 
occupation, non-English speaking home, low self-esteem, poor school attitudes, 
alcohol and/or substance abuse problem, and negative police involvanent. This 
type of information was not available for the Earn and Learn students. 

Study Design 

This study's purpose was to analyze the effects of the current Earn and Learn 
Program on students' middle school experience. It is based on a pretest- 
posttest experimental/control group design. As already stated, the target 
population was Earn and Learn students who graduated eighth grade either in 
1988-89 or 1989-90. The control group consisted of students who were referred 
to Earn and Learn but who either chose not to attend or for whom there were no 
program slots available. 

Measures 

Numerous measures were used to analyze Earn and Learn outcomes. These measures 
can be categorized according to three outcome types: 

1) Academic Variables including standardized achievement data and 
grades ; 

2) Academic Motivation Variables including absences and tardies; 

3) Social Behavior Variables including suspensions. 



ERLC 



Student cumulative folders, the district conputer database and other central 
office files were used as sources to collect this information. For all 
measures, sixth grade data were used as pretest scores and eighth grade data 
were used as posttest scores. 

In a study of this type, one inevitably has to deal with missing data. 
Sometimes test scores or other information were missing because students in the 
sample had previously been enrolled in another district. Sometimes report card 
data were incomplete. Therefore, the number of students included in each 
comparison varies slightly depending on the pretest/posttest data available. 

Middle School Results 

Results are reported according to the three types of outcome variables under 
study: Academic variables, academic motivation variables and social behavior 
variables . 



Academic Variables . Two types of academic variables were analyzed: 
Achievement Test (CAT) scores and grades. 



California 



Table 2 shows the average pretest/posttest scale scores on the C.A.T. of the 
Earn and Learn student group ccnpared to the control group for reading, English 
language and mathematics. An analysis of covariance was carried out to 
determine if differences between the two groups were statistically significant. 
This technique adjusts for differences, if any, that are observed in pretest 
scores. Adjusted posttest scores (noted as Adj. SS) are also included in Table 
2. 



Table 2 Average Pre/Post Scale Scores: California Achievement Test 

Posttest 

& Xffi 33 Mi. SS SiQ. 



N 



Pretest 
XSS 



IDEAL READING 



Earn and Learn 31 721 24 
Control 22 725 30 



751 
750 



24 
21 



752 
748 



n.s.* 



TOTAL LANGUAGE 



Earn and Learn 33 690 30 
Control 27 687 40 



719 
710 



28 
38 



718 
711 



n.s. 



TOiaL MATHEMATICS 



Earn and Learn 33 730 36 
Control 26 714 37 



758 
746 



36 
38 



752 
753 



n.s. 



In all three subject areas, there were no significant differences at the .05 
level between the groups at the end of eighth grade when posttests were 
administered. 



*n.s. group scores are not significantly different. 
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Table 3 shews pretest/posttest data for average GPAs based on report card 
grades in reading, language arts, mathematics, science and social studies. 
Grade point averages for each student were derived by averaging grades for the 
three marking periods in the sixth and eighth grade, respectively. 

Table 3 Average Pre/Post Grade Point Average by Subject Area 



N 



Pre 
XGPA 



SD 



Poet test 
XGPA SD Adi.GPA Sig. 



READING 



Earn and Learn 
Control 



23 
22 



1.75 
1.80 



.89 
.94 



1.93 
1.73 



.88 
.77 



2.94 
1.72 



n.s. 



LANGUAGE ARTS 



Earn and Learn 
Control 



26 
26 



1.98 
1.74 



.79 
.77 



1.96 
1.52 



.77 
.70 



1.91 
1.57 



p = .07 



MATHEMATICS 



Earn and Learn 
Control 



29 
29 



2.47 
1.93 



.79 
.73 



2.59 
1.89 



.84 
.88 



2.44 
2.04 



p - .06 



SCIENCE 



Earn and Learn 
Control 



24 
26 



1.99 
1.91 



.66 
.70 



2.04 
1.61 



.94 
.99 



2.02 
1.64 



n.s. 



SOCIAL STUDIES 



Earn and Learn 
Control 



22 
22 



1.62 
1.58 



.91 
.86 



1.68 
1.54 



.94 
.73 



1.67 
1.55 



n.s. 



Although there were no significant differences between the groups at the .05 
alpha level, as tested by an analysis of covariance, there were trends toward 
significance in two subject areas — language arts and mathematics. In 
language arts, the mean GPAs suggest that while Earn and Learn students 
maintained an average GPA of approximately a "C" average, the control group's 
GPA decreased from slightly under a "C" average to midway between a "D" and "C" 
average. For mathematics, the GPAs indicate that the control group's 
performance declined by .04 from pretest to posttest while the Earn and Learn 
group improved by .12. 
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Acadanic Matxvation Variables . Table 4 shows the percent of students falling 
into three categories of absence types: a) those students missing school 0-10 
half days; b) those students missing 11-20 half days; and c) those students 
missing 21 or more half days. The highest number of half days missed in this 
category was 56 half days (2 students for sixth grade) and 99 half days (1 
student) for eighth grade. It is not possible to document whether some of 
these more extreme cases were due to extended sickness or truancy. The 
variability of the data, however, is clear. 

Table 4 Percent of Students Who Were Absent in Middle School 

Pretest Poettest 

. [Nanber of Half Days ■ , , Number of Half Days 

N 1 0-10 I 11-20 I 21 or I Sig. I I 0-10 1 11-20 I 21 or I Sig. 

more more 



Earn & Learn 44 50% 25% 25% 
Control 35 28% 26% 46% 



n.s. 



29% 32% 39% 

20% 14% 66% p=.05 



A chi square test of significance applied to absence data found no significant 
differences between the groups at sixth grade. However, there was a significant 
difference between the groups in favor of Earn and Learn at eighth grade (X 2 - 6.02, 
df = 2, p = .05). While 39% of Earn and Learn students missed 21 or more half days, 
representing approximately 2 or more weeks of school, two-thirds or 66% of control 
students missed 2 or irore weeks of school. It should be pointed out that when an 
average half day absence rate is computed for Earn and Learn eighth grade, that figure 
is 21.6 which is approximately the same as the district average rate at eighth grade 
of 21.5. 

Table 5 Percent of Students Who Were Tartly 



Pretest 



Poetteet 



N 





Nnter 


of Tardies 


9 




Nuuber c 


pf Taxdi< 


ps 


2 or 
less 


From 
3-12 


13 or 
more 


Sig. 


2 or 
less 


From 
3-12 


13 or 
mare 


Sig. 



Earn & Learn 35 31% 29% 40% 

Control 32 37% 41% 22% 



n.s. 



9% 
16% 



40% 
37% 



51% 
47% 



n.s. 



Table 5 shows the percentage of students falling into three categories of 
tardy types: a) those students tardy 2 times or less; b) those students tardy 
from 3 to 12 times, and c) those students tardy more than 12 times. The 
highest number of tardies was 75 at sixth grade and 74 at eighth grade. A chi 
square showed no significant differences between the groups at pretest and 
posttest. 



Social Behavior Variables, Suspension data were also analyzed. Table 6 shows 
these data. With one exception, no students in the control and Earn and Learn 
groups were suspended in sixth grade. In eighth grade, differences between 
groups as assessed by means of a chx square test were not significant; that is, 
approximately 12 percent of both the Earn and Learn and control groups had been 
suspended one to two times. 



Table 6 



Percent of Students Suspended, 0, 1, or 2 Times 



Pretest 



Posttest 





N 


0 


1 


2 


0 


1 


2 


Earn and Learn 


43 


98% 


2% 


0% 


89% 


9% 


2% 


Control 


39 


100% 


0% 


0% n.s. 


87% 


10% 


3% 



Achievement of Target Goals . For the 1990 eighth graders in this study, data 
were available to analyze achievement of target goals. Since control students 
were not part of the program, this analysis only includes Earn and Learn 
students. Five target goals (e.g., homework completion, on time for class, 
etc. ) were identified and agreed upon by each middle school student and his/her 
teachers. Daily records of the extent to which each participant exhibited the 
desired behaviors were kept and were used to determine if the student was to 
participate in the Earn and Learn activities each day. Daily records were 
relayed to the computer center and cumulative records of the percents of goal 
attainment for each participant were derived. 

Table 7 is a frequency distribution of the percent of time that goals were met 
by the program participants. 

Table 7 Frequency Distribution of Percent of Goal Attainment 
By Eighth Graders, Earn and Learn - 1989-90 



% of Time 


N 


% 


0-9 


0 


0 


10 -19 


0 


0 


20 -29 


0 


0 


30 -39 


0 


0 


40 -49 


1 


4.55 


50 -59 


1 


4.55 


60 -69 


4 


18.18 


70 -79 


6 


27.27 


80 -89 


1 


4.55 


90-100 


9 


40.90 



Number of Goals: 110 
Number of Students: 22 

Of the 22 students for whom data were available, 41% of the goals were 
exhibited from 90-100% of the time, 5% from 80-89% of the time and 27% from 
70-79% of the time. Seventy-three percent of the goals were attained from 70 
to 100% of the time. 
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LONG-TERM EFFECTS OF THE EARN AND LEARN PROGRAM 
ON HIGH SCHOOL PERFORMANCE 

Study Design 

This substudy' s purpose was to investigate the long-term effects of the Earn 
and Learn program on high school performance. It is based on a pretest-postest 
experimental/control group design. The target population for this substudy was 
61 Earn and Learn students who graduated from eighth grade between 1984 and 
1986/ representing the high school graduating classes of 1987-88, 1988-89 and 
1989-90. Students were either enrolled in Earn and Learn beginning in seventh 
grade or in eighth grade. Fifty-nine percent continued in Earn and Learn as 
freshmen, of this population, 26% were white boys, 11% were white girls, 31% 
were black boys, 23% were black girls, 2% (n=l) were Asian boys and 2% (n=l) 
were Hispanic boys. It should be noted that this Earn and Learn group had 8% 
more non-minority students than the more recent target group in substudy 1. 
This group represents students from all four middle schools. 

The control group consisted of students who were referred to Earn and Learn, 
but who either chose not to attend or for whom there were no program slots 
available. 

Comparability of groups prior to intervention was assessed by means of at test 
on measures of reading, English language, mathematics and absence rate. There 
were no significant differences between the groups in subject areas although 
the means of the Earn and Learn group tended to be slightly higher than the 
control group. Both the Earn and Learn and control group were comparable in 
terms of sex/race characteristics. 

Measures 

Similar to the first substudy, the measures for this second substudy also can 
be categorized according to three outcome types: 

1) Academic Variables including standardized achievement data, English grades, 
credits, GPA and GPA percentile. 

2) Academic Motivation Variables including absences and graduation rate. 

3) Social Behavior Variables including suspensions. 

The district computer duuabase, central office files, and Earn and Learn files 
were used as sources to collect this information. Data were also collected 
from the high school with assistance from the Data Processing Department, the 
Office of the Registrar and Information Services. 

Longitudinal Findings 

Areytemic Variables . Although the focus of this substudy is on the longitudinal 
effects of Earn and Learn on high school performance, achievement test data 
from eighth grade were available. Therefore, these data were analyzed along 
with high school informtion. 
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Table 8 shews sixth and eighth grade C.A.T. achievement test data for reading, 
English language and mathematics . An analysis of covariance was used to 
compare group means after any adjustment for differences between the groups. 
There were no significant differences between the groups. 

Table 8 Average Pre/Post Scale Scores: California Achievement Test 

Pretest Posttest 



Earn and Learn 
Control 



N 



61 
51 



xss 



493 
480 



SD 



60 
50 



XSS 



SD Adi.SS 



READING 
554 
538 



66 
64 



Siq. 



549 
545 



n.s. 



Earn and Learn 
Control 



Earn and Learn 
Control 



61 
50 



61 
50 



506 
501 



487 
473 



50 
51 



42 
40 



ENGLISH LANGUAGE 
563 60 
565 64 

MATHEMATICS 
547 57 
534 59 



561 
567 



542 
541 



n.s. 



n.s, 



Table 9 shows data from the Degrees of Reading Power (DRP) , which is a 
standardized achievement test in reading administered to high school students 
at the end of their freshman year. The class of 1988 is not included in this 
analysis because the DRP was not administered to this group of students. The 
DRP provi ies scores in terms of Normal Curve Equivalents (NCE's) which range 
from 1 to 99 with 50 representing the national norm. Since a reading pretest 
score was available (the C.A.T reading subtest score at sixth grade), an 
analysis of covariance was used to adjust posttest scores for any pretest 
differences. No significant differences were found between the groups. 

Table 9 Average Normal Curve Equivalents for Degrees of Reading Power 
Test 

N XNCE SD Adi.NCE Siq. 



Earn and Learn 30 
Control 29 



58 
51 



16 
15 



55 
54 



n.s. 



Table 10 provides data on students' average English grades in each of four high 
school years. English grades were analyzed because all students must take 
English from freshman to senior year. While GPA can only be used to compare 
students who graduated, analysis of English grades allows for analyzing 
students who may not have attended for all four years but were present one or 
more years. Grades range from "A" to "F" with point values ranging from 5.0 to 
0.0. See Appendix C for definitions of point values. Although average GPAs 
are slightly higher for the Earn and Learn group, it must be remembered that, 
in general, the Earn and Learn group tended to be higher on C.A.T. scores at 
the pretest but not significantly so. No significant differences were found 
for any of the high school English GPAs as measured by a Mann-Whitney U test. 
This statistic was used because of certain characteristics of the English GPA 
data which did not lend itself to at test. This test assigns a mean rank to 
each group and evaluates the difference. These ranks are reported in Table 10. 



Table 10 Average Grade Point Average and Mean Rank: 
High School English 



N 



XGPA 



SD 



X Rank 



FRESHMAN ENGLISH 



Earn and Learn 57 
Control 49 



2.00 
1.89 



.85 
.84 



54.21 
52.67 



n.s. 



SOPHOMORE ENGLISH 



Earn and Learn 54 1.94 .79 

Control 47 1.64 .97 



54.96 
46.45 



n.s. 



JUNIOR ENGLISH 



Earn and Learn 49 1.69 .91 

Control 41 1.48 .86 



47.35 
43.29 



n.s. 



SENIOR ENGLISH 



Earn and Learn 44 1.84 .75 

Control 39 1.78 .74 



43.38 
40.45 



n.s. 



For those students who graduated, credit hours, GPA, and GPA percentile were 
analyzed. Table 11 shows credit hour data. The average number of credit hours 
for Earn and Learn students was 50.02 while control group students accumulated 
47.38 credit hours. A t test found significant differences between the 
groups in favor of Earn and Learn (t = 2.31, df = 75, p = .01). Earn and Learn 
students had more credit hours accumulated at graduation than control 
students . 

Table 11 Average Nmfcer of High School Credit Hours 



N X Credit Hours SD Siq. 

Earn and Learn 45 50.02 5.31 

Control 32 47.38 4.32 p = .01 

In Table 12, GPA data are summarized. 

The average GPA for both Earn and Learn and control group students represents a 
"C" average and are not statistically different (t test) . 

Table 12 Average High School Grade Point Average 

N XGPA SD Siq. 

Earn and Learn 45 2.08 .52 

Control 32 1.98 .52 n.s. 
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The high school also assigns a percentile rank to each student's GPA. For 
example, for the class of 1989, a GPA of 2.27 was at the 26th percentile or, a 
student with that GPA performed better than 26% of his/her class. The 
distribution of percentile ranks is shown in Table 13. IWo-thirds or more of 
students in both groups graduated in the bottom quarter of their class. A chi 
square test showed no significant differences between the groups. 

Table 13 Percent of Students in Each High School GPA Quartile 

Siq. 



n.s. 



Academic Motivation Variables . Two types of academic motivation variables were 
investigated: absence rate and graduation/dropout rate. Since eighth grade 
data were also available, it is reported here along with high school data. 
Table 14 shews the percentage of students falling into three absence categories 
for middle school: a) those students missing 0-10 half day absences; b) those 
students missing 11-20 half days; and c) those students missing 21 or more 
half days. 

Table 14 Percent of Students Who Were Absent in Middle School 







1-24 


24-49 


50-74 


75-99 




N 


%ile 


%ile 


%ile 


%ile 


Earn and Learn 


45 


67% 


29% 


4% 


0 


Control 


31 


71% 


23% 


6% 


0 



N 



Nun 


pber of Hall 


E Days 


0-10 half 
days 

34% 
10% 


11-20 
half days 

25% 
29% 


21 or more 
half days 

41% 
61% 



Sig. 



Earn & Learn 59 34% 25% 41% p = .009 

Control 51 



There was a significant difference between the groups as measured by a chi 
square test (X 2 - 9.36, df = 2, p = .009). A greater number of control 
students were absent for 21 days or more compared to Earn and Learn students, 
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Table 15 shows the percent of students falling into three absence categories 
while in high school. Although in eighth grade/ there was a difference between 
Earn and Learn and control students, there were no significant differences in 
absences during high school years. 



Table 15 



Earn and Learn 
Control 



Earn and Learn 
Control 



Percent of Students Absent Half Days in High School 
Number of Half Days 



0-10 half 
N days 



54 
46 



50 
43 



11-20 half 
days 



21 or more 
half days 



39% 
24% 



32% 
16% 



26% 
35% 



22% 
28% 



FRESHMAN YEAR 

35% 
41% 

SOPHCMDKE YEAR 
46% 
56% 



Siq. 



n.s. 



n.s. 



Earn and Learn 40 32% 
Control 35 23% 



23% 
29% 



JUNIOR YEAR 



45% 
48% 



n.s. 



Earn and Learn 34 32% 
Control 27 22% 



12% 
30% 



SENIOR YEAR 



56% 
48% 



n.s. 



Graduation rate was also evaluated. Graduation is explained in the literature 
on dropouts as those students of a given cohort who graduated as compared to 
those students who dropped out. Students who transferred to another school are 
not considered dropouts. Table 16 shows the percentage of students who 
graduated from the cohorts (classes) of 1988, 1989 and 1990 in contrast to 
those who dropped out. This analysis excluded students who transferred to 
another school or who were still enrolled at the high school at the time of 
data collection. No significant differences between the groups were found by 
means of a chi square test. Approximately 80 percent of both groups graduated 
from high school; the overall high school rate for the most recent graduating 
classes is 92%. Looking at the data relative to dropouts, then, approximately 
twice as many Earn and Learn students dropped out as compared to the overall 
high school rate (17% vs. 8%). 
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Table 16 Percent of Students by Graduation/Dropout Category 

N Graduates Dropouts sig. 

Earn and Learn 54 83% 17% 

Control 40 80% 20% n.s. 



Social Behavioral Variables . Table 17 shows the percentage of students 
receiving suspensions in each of the years they were enrolled at the liigh 
school. Between 13% and 22% of Earn and Learn students received suspensions; 
between 11% and 20% of control students were given suspensions. For students 
who were suspended, the number of suspensions ranged from 1 to 11. A chi 
square test found no significant differences. 



Table 17 Percent of Students by Suspension Category 



Earn and Learn 
Control 



Earn and Learn 
Control 



Earn and Learn 
Control 



Earn and Learn 
Control 



N 



56 
46 



52 
43 



41 

35 



33 
26 



0 Suspensions 

FRESHMAN YEAR 

84% 
80% 



1 or mure 
Suspensions 



SOROCHE YEAR 



87% 
74% 



78% 
89% 



100% 
100% 



JUNIOR YEAR 



SENIOR YEAR 



16% 
20% 



13% 
26% 



22% 
11% 



0% 
0% 



Sia . 



n.s. 



n.s. 



n.s. 



n.s. 
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PARENT FEEDBACK ABOUT EARN AND IEARN 



Along with student data, feedback from parents was solicited. For this part of 
the study , Earn and Learn parents were surveyed regarding their attitudes about 
this program. At parent meetings in February and April, 1991, questionnaires 
were distributed to parents of the ninth graders and middle school students, 
respectively. Questionnaires were mailed to those who were not present at 
these meetings. A 19% response rate was obtained for middle school parents. 
The results of this middle school survey should be interpreted with caution due 
to the low response rate. An 82% response rate was obtained for parents of 
high school students. The children of some of these parents are part of the 
study on middle school effects. 

The distribution of responses on a four point scale, from "strongly disagree" 
to "strongly agree" as well as the average score for each item are shown in 
Table 18. 

Feedback from parents of current middle school students 

Overall, with the exception of two questions, 50% or more of parents selected 
either agree or strongly agree throughout the survey. Parents indicated the 
most dissatisfaction (disagree or strongly disagree) for item 3 and item 22d: 
staff response to student behavior (54%) and the importance of summer canp 
(73%) . The parents were asked to evaluate three major areas of the Earn and 
Learn Program; (1) staff, (2) program administration, and (3) program services. 
The most positive responses occurred within the area of Program Services; 50% 
to 92% responded on the high end of the scale. 

Staff . Responses to questions pertaining to Staff indicated parents feel 
most positive in the area of communication between themselves and the work site 
staff. Seventy-five percent of the parents indicated satisfaction with the 
ways in which their child's progress is reported to them; 67% felt that 
concerns about their child are communicated to them in a timely nanner. 
Parents were less positive toward the staff's role at the work site, in regard 
to such things as the staff helping to create a positive atmosphere (50%) and 
the response of staff to student behavior (45%). 

Program Administration . The parents' responses in the area of Program 
Administration were distributed mostly under agree or disagree, with few 
responses at either end of the scale. Sirty-four percent of the parents felt 
that administrators are receptive to their concerns while only 50% felt that 
administrators are consistent and fair in dealing with student behavior. 

Program Services . Parents provided the most positive responses to the portion 
of the survey that focused on Program Services . Parents seemed particularly 
pleased with four areas: feeling that the program motivates students to 
inprove in school (92%), feeling welcome at the work site (92%), feeling the 
parent meetings are helpful (91%) and feeling satisfied with Earn and Learn 
social work services (88%). Parents seemed less satisfied with their child's 
experience in summer camp (66%) . Half of them did not necessarily feel that 
the program inproved school attendance. 
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Parents were also asked to rate four components of Earn and Learn on a four 
point scale from "Not at all Important" to "Very Important." The irajority of 
parents rated the first *-hree ccrponents as important or very iirportant: work 
component (64%), homework lab (91%) and student contact with middle school 
advisors (92%). 

Sumner camp, however, was rated less favorably with no parent feeling it was 
very inportant and 73% feeling it was either not at all or somewhat important. 

Parents were also asked seme open-ended questions. The caiments in regard to 
the benefits of Earn and Learn were as follows: 

- "Motivation do all their homework and to improve their grades." 

- "A sense of accomplishment." 

- "It lets than know that there are some people who care about their 

perforrance." 

- "Working with others/team building; learning to respect authority; 

discipline; pay - responsibility for cwn decisions, helps set 
priorities, gives them a taste of real life." 

In response to how Earn and Learn can be improved, most parents felt that the 
philosophy of the program itself is quite good, but that it is not being 
implemented properly: 

- "Earn and Learn can be helpful, if the business of the children is 

taken care of." 

- "Concept good , not being worked properly." 

- "Follcw the original guidelines set and really work to motivate the 

children." 

- "I don't see much evidence of Earning to learn. I get the impression 

that the work and pay overshacfc** the whole objective of Earn and Learn 
which is to promote the institution of education." 

The only comment under program services was: 

- "I do not think summer camp should be mandatory!" 
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Feedback from Parents of Students in Ninth Grade 



Overall , with the exception of one question, 64% or more of parents selected 
either agree or strongly agree throughout the survey. Parents responded 
somewhat less positively (disagree or strongly disagree) to three items: staff 
response to student behavior (36%), administrators 1 consistency in dealing with 
student behavior (35%) and summer camp (46%). Parents were asked to evaluate 
three major areas of Earn and Learn: Staff, Program Administration and Program 
Services. The most positive responses were with respect to Program Services. 
From 66% to 92% responded at the high end of the scale. 

Staff . Responses to questions pertaining to Staff were very positive in all 
areas. Parents felt most positive with the ways in which their child's 
progress (84%) and concerns regarding their child (79%) were reported to them, 
and with the way the staff creates a positive work atmosphere (79%). Parents 
seemed less satisfied with the way staff responds to student behavior at the 
work site, with 36% indicating they disagreed or strongly disagreed with how 
appropriately staff responds. 

Program Administration . Under Program Administration , the overall response, 
while remaining positive, was less so than the responses in other sections of 
the survey. Seventy-two percent of parents felt that the Program Director 
provides positive leadership within Earn and Learn, that the Earn and Learn 
administrators are viewed as approachable by their child as well as being 
receptive to parent concerns. Two-thirds of parents felt administrators are 
consistent and fair in dealing with student behavior. 

Program Services . Parent responses to Program Services were most positive in 
four areas: 92% were satisfied with the Earn and Learn social work services, 
84% felt comfortable at Earn and Learn parent activities, 83% felt the parent 
meetings were helpful and 83% were satisfied with the student bus service. 
There were two areas in which the responses were less than 80% . Sixty-six 
percent of parents felt Earn and Learn has helped their child to complete more 
homework and 67% felt Earn and Learn has helped their child to improve school 
behavior. 

When asked to rate the four components of Earn and Learn (on a four point scale 
from "Not at all Important 11 to "Very Important 11 ) , all parents (100%) felt that 
the first three components, work conponent, homework lab and student contact 
with middle school advisors, were important or very important. The fourth 
component, summer camp, was rated much less favorably with only 54% feeling it 
was important or very important. 

When asked how the Earn and Learn Program can be improved, responses were: 

- "Focus more on kids adjustment and coping than on trying to change 

parents . " 

- "It could be expanded. " 
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Parents reported that the benefits of Earn and Learn Program for students were: 

- "Help with behavior" 

- "Improved self-esteem" 

- "Improved self -discipline" 

- "Learning to be more responsible" 

- "Learning that work, etc, is rewarded" 

- "Develop self-confidence" 

- "Growth" 

- "Earn and Learn program has helped my sons with their grades and 

attendance" 

- "Help them in school" 



SUMMARY OF FINDINGS 



Two studies were carried out to evaluate the Earn and Learn program. One study 
focused on the effects of this program on students' performance in middle 
school. The second study investigated long-term effects of Earn and Learn on 
students 1 high school performance. Variables under study included standardized 
achievement data, report card grades, absences, tardies, suspensions, credit 
nours and graduation rate. Parent feedback about tne Earn and Learn Program 
was also sought. Statistical techniques of group comparison and pre/post 
testing were used to determine effects of the program. 

Overall, for most variables under study, no differences were found between Earn 
and Learn and control groups for achievement test data, report card grades, 
tardies, suspensions and graduation rate. Major outcomes were: 

- A significant proportion of Earn and Learn students had fewer absences 
than control students at the end of eighth grade. 

- Earn and Learn students tended to get higher grades in language arts 
and mathematics in eighth grade than a comparison group of control 
students but were not significantly different. 

- In high school, Earn and Learn students earn significantly more credit 
hours at the time of graduation than control students. 

Overall, parents responded positively to most aspects of the Earn and 
Learn Progiam. Responses were least favorable with respect to 
communication between staff and students and the summer camp 
experience . 



CONCLUSIONS AND IMPLICATIONS 

Earn and Learn f s stated purpose is to improve school performance by motivating 
students to improve school attendance, behavior, classwork and homework. Study 
findings suggest tnat Earn and Learn improved student attendance in the middle 
school years, but had little effect in high school. Although one of the 
program goals is also to improve behavior, Earn and Learn students did not 
enter with severe behavior problems that would be cause for suspension. The 

- at sixth grade indicated that only one student nad been suspended in sixth 
y .aae prior to program study. 
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Of particular importance is whether Earn and Learn 's focus on attendance, 
behavior, homework and classwork contributes to improved academic performance. 
Study results indicate that students earned more credit hours and tended to 
perforin higher academically than controls; however, there appears to be no 
significant effect on achievement test scores, grades or GPA. Finally, the 
Earn and Learn Program does not appear to improve graduation rate or, 
conversely, decrease the number of dropouts as compared to control groups of 
students. Although Earn and Learn students enter the program with some 
characteristics associated with dropouts, their attendance rate is quite good, 
suspension rate is low and many students perform at above average levels. 

As a result of these findings, the program has been restructured with more 
explicit selection criteria and procedures as well as a redesigned work 
experience and new community service and career awareness components. 
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APPENDIX A 

CRITERIA FOR SELECTING STUDENT PARTICIPANTS 
IN EARN AND LEARN 

(As printed in Program Handout) 



When a child needs assistance in relating to a child or in enabling a child to 
benefit from instructional activities, a number of district resource persons 
are available for consultation: fellow teachers, administrators, special 
services and ancillary personnel. 

If these initial consultations are not sufficient to improve the situation and 
it is the professional judgement cf the staff that the situation calls for some 
special intervention, the school counselor in consultation with appropriate 
special services personnel or the entire "Intake" teaml evaluates the 
alternatives available and appropriate to meet the needs of the child. Earn 
and Learn is one of these alternatives. While the stimuli for evaluating a 
situation vary, (repeated confrontations with peers and teachers, self 
initiated withdrawal from the group, unwillingness to make consistent attempts 
to learn), the criterion for recommending that a student participate in the 
Earn and Learn program is the staff interpretation that the student needs a 
systematic incentive program within a group setting in order to gain a new 
orientation and level of personal integration. A record of the recommendations 
resulting from the consultations or Intake evaluations is maintained by the 
school counselor/coordinator. 

Once selected for the program, a specific set of student behavioral goals are 
defined by the teachers and reviewed with the student and parents before 
participation can begin. Criteria for evaluating goal accomplishment are 
derived for each of the target behaviors. 

Evaluation of goal accomplishment is a continuous process, initiated and 
monitored by the counselor/coordinator. Teachers are asked to select new goals 
for participants when a high level of success is achieved. 2 

x.i selecting children to participate in the program, the existing iace and sex 
composition of the program is taken into consideration so that segregation by 
race or sex will be avoided* An attempt is made to have an even number of 
seventh and eighth graders. Some sixth graders are accepted under special 
circumstances. A criterion describing an acceptable student participant/staff 
ratio has not been established to date, since the allocation of space for 
project activities is a present limitation. 



1 Multidisciplmary team that operates at each school, meeting regularly, 

for the purpose of dealing with staff concerns regarding individual 
students. 

2 90% success over a tri-semester . 



REFERRAL FOR EARN AND LEARN 



APPENDIX B 



Please carplete and return to Earn and Learn Advisor 



Date 

Student's Name School 



Race Sex Grade and Section 

Homeroom Teacher Referring Person 

Position 



Place a check (x) next to those behaviors frequently exhibited by student. 

Inccnplete homework 

Inccnplete classwork 

Tardy to class 

Tartly to school 

Missing materials for class 

Inappropriate conversation in class 

Disruptive movements in class 

Does not participate in class activities and/or discussion 

Does not follow directions 

Does not pay attention in class 

Seeks inappropriate attention from adults 

Seeks inappropriate attention from peers 

Withdraws from peers 

Problems relating with peers 

Problems relating with adults 

Aggression toward objects 

Aggression toward peers 

Aggression toward adults 

Denies responsibility for actions 

Preoccupied with family problems 



Please describe: 



« 
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HIGH SCHOOL GRADING SYSTEM 



to * 



Student Frogreee Wooo r tB are prepar ed four ttmee eeeh y«ar. Thoeo naoerte ere iiajplimanisd toy ether u a ir eepi 

ditionat information should be shared with you. 

You may obtain additional information anytlmo by consulting with Individual teachers or wtlh your student' I 
and taiapnona number appear on ma Student Frogreea Hajajajrt 



i a mo n ovor \ 



> believe ad- 



Current and cumulate* credits and grade point averegec t ppaar on flrat 

Gr ado point average (OF A) la dotorminod by averaging t orooostr morfca of A. A + C C. C - . 0 

carrying ono or mora than ono credit, except physical educa t ion. Foam v oluoe am UaHd unde r et artU wej 
O S point; advanced placement marks are w ighted by ■ doing 1.0 poem. 



Your counselor's name 



0, F, end f for a* CTHS couraea 
by edding 



Marking Symbol* 



SYMBOL 


DEFINITION 


VALUE 


SYMBOL 


OfFmmoN 


VALUE 


A 


Superior 


4.0 


I 




0.0 


A - 


Superior 


3,7 


P 


Feee a* PoneJFoM Ceurae 


Credit 


8 + 


Excellent 


3J 


FF 


FeU in FeoaVFad Course 


NA 


B 


Excellent 


10 


a 




baai NA 


• - 


Excellent 


2.7 


AO 




NA 


C* 


Average 


2J 


LA 


Lets Bob— 1 - A 


NA 


C 


Average 


2M 


L» 


La*b*M >B 


NA 


C- 


Average 


1.7 


LC 


Lata brtrant • C 


NA 


0* 


Below Average 


1.3 


LD 


Lota BnawM - 0 


NA 


0 


Below Average 


1.0 


LF 


L^se Bneront * FoaJeej 


NA 


F 


Failed* 


04 









• in order for a student to receive a grade of F in a aval em mat haa dltf erent group* lor ability and a^chaavoaaaavt. the r e aeon lor the F should be cioeriy 
known. The F grade mutt be explained. 



F1 
F2 
F3 
F4 



Lack ol attendance 
Failure to turn in required work or 
Too difficult for the student 
Other narrative to poronta 
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